This is where the paradoxical paradoxes of family dynamics come in. It's a thing where a grandfather's influence is so profound, his grandkids can't even remember their real parents.
Read more about the Paradox of Identity.
But don't get us started on Family Ties that Bind.